Multi-provider rebalancing: AWS-scope gate, scope AWS-locked helpers, non-AWS eval cases (v0.6.4)#123
Merged
Merged
Conversation
…l cases (v0.6.4) StorageOps serves all S3-compatible object storage, not only AWS. A self-review found the recent deterministic helpers leaning AWS-specific (IAM/notification validators) while the multi-provider scaffolding stayed intact. This corrects the lean and adds a guardrail. - New gate scripts/provider_scope_check.py (wired into ci.yml): flags any helper that hardcodes AWS-only identifiers (amazonaws.com / arn:aws:iam) without being provider-parameterised (--provider) and without an explicit "AWS-specific" scope declaration. Turns silent AWS-lock into a CI failure. Unit-tested. - Scoped the 3 AWS-locked helpers honestly: notification_config_analyzer.py, notification_target_policy_validator.py, cross_account_access_validator.py now declare AWS-specific in their docstrings, emit "model": "aws" in every result, and point at provider-differences (OSS RAM / COS CAM / BOS differ). SKILL.md "Run when" lines note the AWS model. Honest scoping rather than faking unverifiable BOS/OSS/COS support. - Rebalanced the corpus with non-AWS cases so the gate stops over-fitting AWS phrasing: bos-multipart-etag-leading-dash (BOS leading-dash multipart ETag vs corruption -> data-consistency) and oss-sigv4-region-required (OSS SigV4 signing-region mismatch -> s3-protocol-compatibility), each with a baseline. Validation: all gates green incl. the new provider-scope gate; 211 pytest + 21 extension tests; 30/30 baselines 100% PASS; routing-corpus recalls both non-AWS cases. Version 0.6.3 -> 0.6.4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhkyBLeDtBTmvTdVBqgCAV
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad78c39359
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| provided. Check your key and signing method. | ||
|
|
||
| Client debug (redacted): | ||
| Authorization: AWS4-HMAC-SHA256 Credential=****/20260621/us-east-1/s3/aws4_request, ... |
There was a problem hiding this comment.
In the new golden case, this line still matches SECRET_PATTERNS in golden_case_validator.py (Authorization ... AWS4-HMAC-SHA256 \S+). I ran python skills/storageops-eval-golden-cases/scripts/golden_case_validator.py skills/storageops-eval-golden-cases/cases, and it fails on this file before the baseline eval step in .github/workflows/ci.yml, so the added case will make CI red until the header is further redacted/renamed or includes an allowed example marker.
Useful? React with 👍 / 👎.
The oss-sigv4-region-required input contained a literal `Authorization: AWS4-HMAC-SHA256 ...` line which the golden_case_validator secret scanner flags even with the credential masked. Replaced it with a credential-free SigV4 scope description that keeps the region evidence (us-east-1 vs cn-hangzhou). Case still routes to s3-protocol-compatibility and its baseline still scores PASS. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhkyBLeDtBTmvTdVBqgCAV
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
StorageOps serves all S3-compatible object storage (AWS S3, MinIO, Baidu BOS, Alibaba OSS, Tencent COS, GCS), not only AWS. A self-review of v0.6.0–v0.6.3 found the recent deterministic helpers leaning AWS-specific (the IAM/notification validators) even though the multi-provider scaffolding (
detect_domainprovider signatures,provider-quirks/, provider-differences references, the 5-provider ETag matrix) stayed intact. This release corrects the lean and adds a guardrail so it can't grow back silently.1. New deterministic gate —
provider_scope_check.py(wired into CI)Flags any helper that hardcodes AWS-only identifiers (
amazonaws.com/arn:aws:iam) without being provider-parameterised (--provider) and without an explicitAWS-specificscope declaration. Turns silent AWS-lock into a CI failure — the same lever used for orphan references in v0.6.3. The genuinely multi-provider helpers (--provideron multipart-ETag, throttle-tuning, access-log) are exempt by construction. Unit-tested (4 cases).2. Scoped the three AWS-locked helpers honestly
notification_config_analyzer.py,notification_target_policy_validator.py, andcross_account_access_validator.pynow:AWS-specificin their docstrings,"model": "aws"in every result, andTheir SKILL.md "Run when" lines note the AWS model too. The honest move is to scope them, not to fake BOS/OSS/COS support we cannot verify — consistent with the project's "mark unverified, don't universalize AWS" stance.
3. Rebalanced the eval corpus with non-AWS cases
So the regression gate stops over-fitting AWS phrasing:
bos-multipart-etag-leading-dash— Baidu BOS leading-dash multipart ETag mistaken for corruption → data-consistencyoss-sigv4-region-required— Alibaba OSS SigV4 signing-region mismatch → s3-protocol-compatibilityEach with a baseline; both verified to route correctly via
detect_domain(routing-corpus top-2).Validation
All gates green incl. the new provider-scope gate; 211 pytest + 21 extension tests; 30/30 baselines 100% PASS; routing-corpus recalls both non-AWS cases. Version 0.6.3 → 0.6.4.
🤖 Generated with Claude Code
Generated by Claude Code